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IKACI 

This  research  study  tested  whether  a  military  adaptation  of  the  Mental  Health  First  Aid 
(mMHFA)  program  changes  knowledge  about  and  attitudes  toward  mental  health  in  the  military, 
stigma  associated  with  mental  health  issues,  and  use  of  appropriate  referral  and  support 
practices  for  helping  someone  in  need.  Phase  one  of  the  study  consisted  of  developing  the 
military  and  veterans  module  for  MHFA.  Phase  two  consisted  of  a  pilot  study  with  four 
armories  from  the  Kansas  Army  National  Guard.  Pilot  study  results  were  mixed,  with  clear 
trends  showing  enhanced  knowledge,  attitudes,  behaviors,  and  confidence  and  reduced  stigma 
among  people  who  took  the  training.  However,  there  were  no  intervention  effects  detected  at 
the  community  level  based  on  a  random  survey  of  guard  members  in  each  armory.  Overall,  the 
study  results  indicate  that  the  military  and  veterans  MHFA  intervention  is  acceptable  and 
feasible  for  National  Guard  armories,  and  is  a  promising  intervention  that  warrants  further 
study  in  a  larger  cluster  randomized  controlled  trial  to  detect  intervention  effects. 
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INTRODUCTION: 


This  research  study  tested  whether  a  military  adaptation  of  the  Mental  Health  First  Aid 
(mMHFA)  program  changes  knowledge  about  and  attitudes  toward  mental  health  in  the  military, 
stigma  associated  with  mental  health  issues,  and  accessing  care  and  support  resources  for 
mental  health  problems  among  military  personnel.  Phase  one  of  the  study  included 
development  of  the  military-specific  MHFA  training  module.  Phase  two  was  a  pilot  study  of  the 
intervention  including  two  groups  of  individuals:  Community  First  Responders  (CFRs)  and 
Kansas  Army  National  Guard  (KSARNG)  service  members.  CFRs  are  individuals  (including 
Guard  Members  in  support  positions)  who  interact  with  Kansas  Army  National  Guard  service 
members  on  a  routine  basis  to  provide  mental  and  emotional  support.  The  pilot  study  included 
four  armory  communities  and  CFRs  in  the  intervention  group  received  training  in  mMHFA. 

KEYWORDS:  National  Guard;  Mental  Health  First  Aid;  Mental  Health  Literacy;  Pilot  Study; 
Veterans;  Mental  Health;  Prevention;  Access;  Stigma 

OVERALL  PROJECT  SUMMARY: 

Phase  1:  Develop  military-specific  MHFA  training  module 

Convene  Expert  Panel  and  form  Adaptation  Taskforce 

An  Expert  Panel  was  convened  in  October  2010  to  advise  on  the  adaptation  of  the  MHFA 
training  module.  Panelists  included: 

LTC  Dennis  McGurk 

Chief,  Research  Transition  Office 

Walter  Reed  Army  Institute  of  Research 

Katharine  W.  Nassauer,  Ph.D. 

Senior  Scientist,  Psychological  Health  and  Resilience 
General  Dynamics  Information  Technology 

United  States  Army  Medical  Research  and  Materiel  Command  (USAMRMC) 

Military  Operational  Medicine  Research  Program  (MOMRP) 

MAJ  Robert  Stinson 

Chief,  Workforce  Support 

Kansas  Army  National  Guard  (KSARNG) 

Jay  Shore,  M.D.  (Co-PI) 

Associate  Medical  Director,  Colorado  Physicians  Health  Program 
Subject  Matter  Expert  and  Psychological  Health  Portfolio  Manager,  Telemedicine  and 
Advanced  Technology  Research  Center  (TATRC),  U.S.  Army  Medical  Research  and 
Material  Command  (USAMRMC) 

The  Adaptation  Taskforce  was  formed  in  November  2010.  There  was  a  change  of  project 
manager  in  June  201 1  and  additional  taskforce  members  added  at  that  time,  which  contributed 
to  a  delay  in  the  project  timeline.  The  taskforce  included  subject  matter  experts  with  specific 
knowledge  of  the  military/veteran  population,  creating  training  curriculum,  the  existing  Mental 
Health  First  Aid  program,  and  the  landscape  of  military  programs  that  provide  support,  training, 
and  assistance  to  service  members  and  their  families. 

Consult  with  expert  panel 
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The  panel  met  by  conference  call  periodically  through  April  2011  to  inform  the  initial  phase  of 
the  adaptation  process.  Additionally,  members  of  the  Expert  Panel  and  other  subject  matter 
experts  were  engaged  throughout  the  research  period  to  advise  on  the  adaptation,  IRB,  and 
research  protocol. 

Develop  military-specific  module  with  Adaptation  Taskforce  and  input  from  Expert  Panel 

The  adaptation  of  the  Mental  Health  First  Aid  curriculum  for  use  with  the  military/veteran 
population  included  the  addition  of  supplemental  material  relevant  to  mental  health  issues  in  the 
military/veteran  population  and  military-specific  videos  and  exercises,  as  well  as  provided 
context  for  the  various  mental  health  issues  when  interacting  with  service  members,  veterans, 
and  their  families.  The  work  products  included  an  adapted  PowerPoint  slide  deck,  instructor 
notes,  and  supplemental  materials  such  as  resource  lists  and  scenarios. 

The  Investigators  maintained  regular  contact  with  the  National  Council  for  Behavioral  Health, 
the  organization  that  leads  the  management,  operation,  and  dissemination  of  Mental  Health 
First  Aid  in  the  United  States,  about  the  adaptation  process  and  timeline,  as  well  as  with  subject 
matter  experts  around  focus  areas  such  as  Traumatic  Brain  Injury. 

A  series  of  3  pilot  trainings  were  conducted  in  Arizona  in  partnership  with  the  Arizona  National 
Guard  and  the  Arizona  Coalition  for  Military  Families.  Pilot  training  participants  included  a  mix  of 
service  members,  family  members,  and  Community  First  Responders.  Feedback  from  pilot 
training  participants  helped  inform  finalization  of  the  curriculum. 

Finalize  mMHFA  program 

The  final  version  of  the  military  adaptation  of  the  Mental  Health  Frist  Aid  program  was 
completed  in  April  2013  and  submitted  to  the  National  Council  for  Behavioral  Health.  The 
adaptation  was  approved  for  use  in  this  study  and  pending  finalization  for  wider  scale  roll  out 
through  their  national  network  of  trainers.  Separate  from  this  study,  a  decision  was  made  by  the 
National  Council  to  shift  the  main  MHFA  curriculum  from  a  12-hour  (2-day)  format  to  an  8-hour 
(1-day)  format.  While  this  change  did  not  impact  the  study  trainings,  which  were  completed 
before  the  shift  took  place,  it  required  some  follow  up  work  on  the  mMHFA  adaptation  to  fit  with 
the  new  8-hour  format. 

These  revisions  were  completed  in  September  2013  and  submitted  to  the  National  Council.  The 
curriculum  was  finalized  and  formatted  into  an  adapted  slide  deck  and  new  instructor  manual. 
Members  of  the  study  team  then  assisted  with  delivering  a  webinar  to  the  existing  network  of 
MHFA  instructors,  to  walk  them  through  the  curriculum  changes  and  specific  guidance  relating 
to  training  focused  on  the  military  and  veteran  population.  This  webinar  was  recorded  and  is 
now  available  to  be  viewed  by  any  MHFA  instructor  that  qualifies  to  deliver  the  mMHFA 
adaptation.  Instructors  must  have  a  direct  connection  with  the  military  (service  member,  veteran 
or  family  member)  and/or  have  training  and  experience  working  with  this  population. 

Develop  survey  tool  and  data  collection  procedures  with  input  from  Expert  Panel 

Two  surveys  were  developed:  1)  a  survey  of  community  first  responders  to  measure  outcomes 
from  the  training  in  comparison  to  a  control  group  of  similar  Guard  and  community  members;  2) 
a  quick  poll  for  KSARNG  members  to  assess  community-level  impacts  of  the  intervention.  CFRs 
survey  and  quick  polls  were  developed  by  the  study  team  in  consultation  with  subject  matter 
experts,  IRBs,  and  KSARNG.  Each  survey  includes  scales  or  single  items  to  assess  the  impact 
of  the  intervention  on  key  constructs  related  to  testing  the  study  hypotheses.  The  surveys  were 
designed  to  be  brief  so  as  to  minimize  the  impact  of  the  study  on  participants,  while  still 
providing  sufficient  preliminary  data  on  program  efficacy  in  preparation  for  a  clinical  trial. 
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Phase  2:  Conduct  a  Pilot  Study  mMHFA  in  collaboration  with  Kansas  National  Guard 

communities 


MHFA  has  been  shown  to  be  effective  in  improving  participants’  knowledge  of  mental  disorders, 
reducing  stigma,  and  increasing  the  amount  of  help  provided  to  others.1'4  Individuals  who  have 
undergone  MHFA  training  show  better  identification  of  mental  health  disorders,2  greater 
likelihood  of  advising  individuals  to  seek  professional  help,3  decreases  in  stigmatizing  attitudes,1 
and  improved  mental  health  in  the  participants  themselves.1  3  These  effects  are  present  in  both 
urban  and  rural  areas,  although  rural  areas  tend  to  show  greater  benefits  of  MHFA  training.2 

The  improved  help  to  persons  suffering  from  mental  illnesses,  reduced  stigma,  and  larger 
positive  effects  of  MHFA  training  in  rural  relative  to  urban  areas  suggest  MHFA,  adapted  for  a 
military  population,  may  have  a  large  impact  on  mental  health  outcomes  among  returning 
National  Guardsmen,  many  of  whom  live  in  rural  areas  where  public  health  education  programs 
that  minimize  the  barriers  to  treatment  are  minimal  or  nonexistent.  With  1.64  million  service 
members  deployed  in  support  of  OEF/OIF,  it  is  estimated  that  approximately  300,000  individuals 
currently  suffer  from  PTSD,  and  another  320,000  from  TBI.5  In  the  military,  nearly  half  of  all 
cases  of  behavioral  health  problems  are  untreated,5  an  issue  compounded  by  a  strong  stigma 
towards  behavioral  health  issues  among  military  personnel.6  For  National  Guard  service 
members,  rates  of  untreated  problems  may  be  higher  given  their  limited  access  to  health  care 
compared  to  other  military  personnel.7  Identifying  an  effective  intervention  to  reach  these 
service  members  has  the  potential  to  have  a  major  impact  on  the  health  of  these  individuals.  In 
addition  to  the  direct  health  benefits,  an  effective  intervention  would  likely  have  broader  societal 
effects  on  employment  and  productivity,  marital  relationships,  and  parenting  and  child 
outcomes.5  The  proposed  project  therefore  has  the  potential  for  a  major  impact  on  both  the 
current  health  and  future  well-being  of  military  personnel  and  their  communities. 

Overview  of  Study  Design  and  Hypotheses 

To  test  whether  the  newly  developed  military-specific  module  to  MHFA  is  a  promising  and 
feasible  intervention  for  replicating  prior  MHFA  research  within  U.S.  military  and  veteran 
populations,  WICHE  conducted  a  comparative  outcomes  trial  with  four  Kansas  Army  National 
Guard  (KSARNG)  armories.  Although  not  a  full-scale  clinical  trial,  the  study  was  critical  to 
assessing  the  acceptability  of  the  new  intervention  among  U.S.  military  and  veteran  personnel 
and  the  feasibility  of  achieving  and  measuring  the  expected  outcomes  using  the  methods 
approved  by  the  Expert  Panel  and  KSARNG.  The  pilot  study  was  designed  to  develop 
preliminary  evidence  of  the  efficacy  of  the  intervention  at  improving  individual  and  community- 
level  mental  health  outcomes.  The  hypotheses  we  tested  include: 

1 )  Relative  to  Community  First  Responders  (CFRs)  that  do  not  receive  military  Mental 
Health  First  Aid  (m-MHFA)  training,  CFRs  that  receive  m-MHFA  training  will  show: 

a.  increased  knowledge  of  mental  health  resources,  self-perceived  ability  to  help 
with  a  mental  health  problem,  and  likelihood  of  using  MHFA  practices; 

b.  more  positive  attitudes  towards  mental  health  help  seeking; 

c.  decreased  stigma;  and 

d.  increased  helping  behaviors  and  referrals  to  mental  health  services. 

2)  Relative  to  National  Guard  personnel  living  in  Armory  communities  where  no  CFRs 
received  m-MHFA  training,  Guard  living  in  Armory  communities  where  CFRs  did  receive 
m-MHFA  training  will  show: 

a.  Decreased  stress  and  stigma; 

b.  Increased  coping  resources  and  knowledge  of  mental  health  resources;  and 

c.  More  positive  opinions  towards  help  seeking. 
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To  test  the  efficacy  of  the  military  module  of  MHFA,  we  employed  a  comparative  outcomes 
design  with  four  matched  armories.  Two  armories  were  assigned  to  receive  the  intervention  and 
two  to  the  control  comparison  group.  The  intervention  and  control  armories  were  matched  on 
rurality,  such  that  in  each  condition  (intervention  or  control)  there  was  one  urban  and  one  rural 
armory.  Prior  to  the  implementation  of  the  intervention,  WICHE  conducted  pre-test  assessments 
with  each  armory,  including  a  survey  of  community  first  responders  (CFRs)  and  an  anonymous 
quick  poll  of  National  Guard  personnel  present  during  drills.  At  four  and  eight  months  post 
intervention  these  same  assessments  were  repeated.  Additionally,  the  CFRs  who  received  the 
training  (i.e. ,  from  intervention  armories)  completed  the  CRF  survey  immediately  after  receiving 
the  MHFA  training.  Pre-intervention  CFR  surveys,  immediate  post-intervention  CFR  surveys  for 
those  receiving  the  training,  and  all  quick  polls  were  completed  in  person  using  paper  and  pencil 
methods.  Four  and  8  month  CFR  follow-up  surveys  were  completed  using  either  an  internet 
survey  emailed  to  participants  or  a  paper  and  pencil  survey  mailed  to  participants  with  a 
stamped  return  envelope,  based  on  respondent  preference.  Figure  1  on  the  following  page 
displays  the  study  design. 

Figure  1.  Study  Design  and  Measurement  Protocol 
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IRB  approval  for  this  study  was  received  in  May  2012  from  Montana  State  University.  The 
protocol  was  then  submitted  for  ORP  HRPO  review  in  June  2012.  The  initial  administrative 
review  resulted  in  a  request  for  documents.  The  response,  revisions,  and  requested 
documentation  were  submitted  and  IRB  reapproval  for  this  study  and  ORP  HRPO  final  approval 
were  received  in  March  2013.  Implementation  of  phase  two  of  the  study  was  then  initiated. 

Identify  training  sites  and  recruit  training  participants 

WICHE  consulted  with  the  Expert  Panel  and  the  KSARNG  to  identify  the  four  Armory 
communities  that  participated  in  the  project,  as  well  as  the  individuals  who  were  recruited  to 
receive  the  training.  Based  on  initial  conversations  with  the  KSARNG,  the  project  identified  two 
rural  sites — Hutchinson  and  Manhattan — and  two  sites  urban  sites — Kansas  City  and  Wichita. 
Once  the  rural  and  urban  sites  were  selected,  the  project  team  randomly  assigned  one  site  from 
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each  pair  to  serve  as  the  intervention  site  and  the  other  sites  to  serve  as  the  matched  controls. 
Figure  2  displays  a  map  of  where  the  study  sites  were  located. 

Figure  2.  Map  of  Kansas  with  the  study  sites  indicated 


Once  the  selection  of  sites  was  complete,  the  Expert  Panel  and  its  KSARNG  representatives 
helped  determine  who  within  and  surrounding  the  KSARNG  Armories  were  CFRs  to  target  for 
study  recruitment.  Initial  discussions  highlighted  the  utility  of  training  individuals  such  as 
embedded  medics,  KSARNG  leaders  who  are  open  to  the  training,  employers  in  the  Armory 
communities,  family  advocates  (especially  those  organizations  that  have  already  expressed  a 
need  for  training),  and  other  community-identified  supports/helpers.  The  Kansas  National  Guard 
identified  approximately  thirty  military  CFRs  for  each  of  the  intervention  communities.  These 
Guard  Members  were  on  duty  at  the  time  and  therefore  funds  had  to  be  budgeted  by  the 
Kansas  Guard  to  account  for  their  time  spent  on  the  project.  In  addition  to  Guard  Members,  we 
sought  to  recruit  an  equal  number  of  civilian  CFRs  for  each  community.  Local  and  statewide 
organizations  and  contacts  were  also  engaged  to  recruit  study  participants.  Recruitment 
activities  included  meeting  with  key  National  Guard  personnel,  email  outreach  and  phone  calls, 
all  using  the  approved  scripts  for  the  study  protocol. 

Registration  for  the  training  was  on  a  first-come,  first-served  basis.  National  Guard  members 
were  prohibited  by  the  IRB  from  receiving  compensation  for  completing  the  surveys,  whereas 
non-National  Guard  CFRs  were  offered  a  $15  gift  card  for  the  completion  of  each  survey.  Based 
on  the  pilot  trainings  conducted  in  Arizona,  which  were  filled  to  capacity  and  offered  no 
compensation  to  participants,  we  did  not  anticipate  any  concerns  with  recruiting  the  maximum 
number  of  civilian  CFRs,  especially  since  the  study  would  provide  a  $150  stipend  to  all  eligible 
civilian  participants.  Unfortunately,  despite  consistent  recruitment  efforts  leading  up  to  the  study, 
it  was  difficult  to  recruit  the  maximum  number  of  CFR  subjects.  While  recruitment  numbers  fell 
below  expectations,  there  were  sufficient  numbers  to  execute  the  study  activities  and  those 
CFRs  who  participated  were  a  good  representative  sample  of  the  intended  audience  for  the 
training  (a  mix  of  military  members  and  civilians  with  support  roles  to  military  personnel  and 
families). 

We  were  able  to  recruit  69  CFRs  into  the  intervention  group  and  107  into  the  control  group  pre¬ 
intervention.  Though  these  sample  sizes  were  promising,  we  were  unable  to  retain  a  substantial 
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portion  of  the  control  group,  with  only  18  individuals  returning  4  and  8-month  follow  up  surveys 
from  the  control  group  (-17%  return  rate)  compared  to  47  individuals  in  the  intervention  group 
(-68%  return  rate). 

In  addition  to  the  CFRs,  KSARNG  personnel  were  randomly  selected  from  the  general 
KSARNG  population  present  at  drill  weekends  during  data  collection  to  participate  in  the  pre¬ 
test,  4-month  and  8-month  quick  poll  surveys.  The  quick  polls  were  completely  anonymous  and 
did  not  necessarily  survey  the  same  KSARNG  personnel  across  each  time  point.  Instead  they 
served  as  a  point  in  time  poll  of  the  Armory’s  general  population.  The  only  criterion  for  inclusion 
was  that  the  individual  be  KSARNG  personnel  assigned  to  one  of  the  four  Armory  sites  chosen 
to  participate  in  the  study.  116  Guard  Members  from  intervention  armories  (where  CFRS 
received  m-MHFA  training)  and  155  from  control  armories  (where  CFRs  did  not  receive  the  m- 
MHFA  training)  were  surveyed  at  pre-test;  104  from  intervention  and  179  from  control  armories 
at  4  months;  and  123  from  intervention  and  63  from  control  armories  at  8  months. 


Organize  and  conduct  trainings 

Trainings  were  organized  in  coordination  with  the  Kansas  National  Guard,  and  held  in  their 
facilities.  A  total  of  four  trainings  were  conducted  from  May  -  July  2013  in  the  intervention 
armory  communities  (two  in  Kansas  City  and  two  in  Hutchinson).  Each  training  had  two 
instructors.  There  was  one  instructor  who  conducted  all  four  trainings  for  continuity,  along  with  a 
second  trainer  that  varied.  Before  each  training,  informed  consent  was  collected  from  all 
participants  and  baseline  survey  data  collected.  At  the  conclusion  of  each  training,  survey  and 
evaluation  data  were  collected.  The  trainings  were  well  received  by  study  participants. 


Additional  study  data 

In  May  2013,  the  quick  poll  surveys  were  administered  to  the  National  Guard  members  within 
each  of  the  four  armory  communities.  Baseline  surveys  were  also  administered  to  CFRs  in  the 
two  control  armory  communities  (Wichita  and  Manhattan).  Informed  consent  was  collected  from 
all  subjects  prior  to  completing  surveys  for  the  study. 


Collect  post-training  data 

Post-training  data  collection,  including  CFR  surveys  and  quick  poll  surveys  at  4-months  and  8- 
months  post-training,  was  scheduled  to  be  collected  in  October  2013  and  February  2014, 
subject  to  the  drill  schedule  of  the  Kansas  National  Guard  for  the  quick  poll.  Due  to  the 
government  shutdown,  drill  for  October  2013  was  canceled,  resulting  in  a  delay  in  the  first  set  of 
post-training  data  collection,  which  ended  up  taking  place  in  December  2013/January  2014. 

Data  collection  included  on-site  quick  poll  surveys  in  all  four  armory  communities  and  an  online 
survey  by  CFRs,  with  contact  made  by  email  and  by  phone.  The  second  set  of  data  collection 
for  the  on-site  quick  poll  surveys  took  place  in  June-July  2014.  The  second  set  of  CFRs  survey 
collection  took  place  in  July  -  September  2014.  Note  that  the  follow  up  surveys  are  referenced 
as  4-  and  8-months  in  this  report. 


Conduct  initial  analyses 

Preliminary  analyses  included  psychometric  analyses  of  the  surveys  to  establish  the  statistical 
properties  of  the  surveys  and  identify  the  best  functioning  items  in  preparation  for  analysis. 
Following  analyses  of  the  measures  we  conducted  a  series  of  statistical  tests  to  check  for 
equivalence  between  urban  and  rural  armories  and  successful  training  implementation. 
Because  we  matched  the  control  groups  on  urban-rural  designation,  it  was  important  for  us  to 
first  determine  whether  it  would  be  necessary  to  account  for  urban-rural  group  differences  in 
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final  analyses.  Independent  samples  t-tests  were  conducted  to  test  for  differences  between 
urban  and  rural  armories  on  stigma,  coping,  knowledge,  and  opinions  from  the  quick  poll  at  pre¬ 
test.  We  focused  only  on  the  quick  poll  data  as  the  data  set  is  larger  and  selection  was 
completely  random,  making  it  the  best  data  set  for  testing  community-level  equivalence  across 
key  variables  that  might  influence  the  program  outcomes. 

The  final  set  of  preliminary  analyses  tested  for  fidelity  of  implementation  with  paired-sample  t- 
tests  of  stigma,  confidence,  knowledge,  attitudes,  and  practice  behaviors  among  CFR  trainees 
pre-test  to  immediate  post-intervention.  This  analysis  was  completed  to  provide  a  check  that  the 
program  was  developed  and  implemented  in  a  fashion  that  could  produce  the  results  expected 
from  prior  research  on  MHFA.  This  is  also  a  form  of  manipulation  check  to  make  sure  that 
observed  differences  in  final  analysis  could  be  traced  to  the  manipulation  (receiving  vs.  not 
receiving  the  training)  as  opposed  to  some  other  external  and  unmeasured  factor. 

CFR  Survey.  The  CFR  survey  included  items  to  assess  confidence,  attitudes,  knowledge, 
stigma,  and  behaviors  using  a  5-point  Likert  scale  from  strongly  disagree  to  strongly  agree.  The 
23  Likert-scaled  items  were  factor  analyzed  to  test  the  scale  construction  and  exclude  poorly 
functioning  items.  First,  principal  axis  exploratory  factor  analysis  was  conducted  using  direct 
oblimin  rotation  to  allow  items  to  load  on  multiple  factors.  This  factor  analysis  identified  the 
primary  factors,  as  well  as  potential  items  to  exclude  from  analyses  due  to  either  not  loading  on 
any  factor  or  loading  significantly  on  multiple  factors.  This  analysis  indicated  that  a  four  factor 
solution  was  strongest,  identifying  scales  for  confidence,  attitudes  toward  mental  health 
treatments,  knowledge  of  mental  health  resources,  and  MHFA  practice  behaviors.  Six  items 
were  identified  for  exclusion  from  these  scales,  and  subsequently  a  second  principal  axis  factor 
analysis  was  completed  on  the  remaining  17  items  using  varimax  rotation  to  minimize  cross¬ 
loading  of  items.  This  analysis  confirmed  the  stability  of  the  four  identified  scales.  Additional 
preliminary  evidence  of  construct  validity  was  confirmed  by  moderately  strong  and  significant 
correlations  between  all  scales  and  the  stigma  item  (r-values  between  .2  and  .5). 

Confidence.  Self-perceived  ability  to  help  people  with  a  mental  health  problem  was  assessed 
with  four  items.  The  items  ask  respondents  about  their  confidence  in:  (1)  recognizing  problems, 
(2)  knowing  what  to  say  and  (3)  do,  and  (4)  their  knowledge  of  mental  illness.  Reliability  for  this 
scale  was  sufficient  (a  =  .78). 

Attitudes.  Attitudes  towards  mental  health  treatments  was  assessed  with  three  items  assessing 
(1)  belief  that  counseling  is  helpful;  (2)  belief  that  medications  are  helpful;  and  (3)  belief  that 
people  can  get  better  with  help.  Reliability  was  sufficient  (a  =  .70). 

Knowledge.  The  majority  of  items  excluded  from  analysis  were  originally  intended  for  a  longer 
scale  assessing  both  knowledge  of  specific  mental  health  issues  and  resources.  However,  two 
of  these  items  were  highly  related  to  confidence  and  cross  loaded  without  clearly  belonging  to 
one  of  the  scales,  two  of  the  items  were  reverse  worded  and  appeared  to  load  on  a  separate 
method  factor,  and  two  of  the  items  did  not  load  strongly  on  any  factor.  The  remaining  four  items 
more  specifically  assessed  knowledge  of  mental  health  resources,  including  knowledge  of  (1)  at 
least  two  resources  in  the  National  Guard;  (2)  at  least  two  resources  in  the  community;  (3)  the 
phone  numbers  for  one  or  more  crisis  line;  and  (4)  at  least  two  trusted  resources  to  refer 
someone  to.  Reliability  for  this  four-item  scale  is  sufficient  (a  =  .76). 

Practice  behaviors.  To  assess  the  use  MHFA  practices,  respondents  were  asked  to  rate 
whether  they  were  likely  to  use  specific  practices.  Practice  behaviors  assessed  include:  (1) 
reaching  out  and  talking;  (2)  asking  about  suicidal  thoughts;  (3)  listening;  (4)  offering  basic  first 
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aid  information;  (5)  assisting  in  seeking  professional  help;  and  (6)  assisting  in  connecting  with 
supports.  Reliability  for  this  scale  was  strong  (a  =  .86). 


Stigma.  Although  no  scale  for  either  public  or  personal  stigma  was  included,  a  single  item  on  the 
CFR  survey  is  used  as  a  proxy  for  measuring  stigma  towards  mental  illness:  “People  are 
generally  supportive  when  a  person  is  experiencing  stress  and  related  mental  health  issues.” 
Due  to  the  positive  wording  of  this  item,  higher  scores  are  indicative  of  lower  levels  of  stigma. 
This  item  most  closely  matches  Link’s  modified  labeling  theory  and  devaluation  subscale  for 
assessing  stigma  and  its  impacts  on  people’s  willingness  to  seek  or  offer  help  8  9.  Link’s 
research  on  devaluation  suggests  that  people  who  respond  more  negatively  to  this  question,  or 
similar  questions,  are  less  likely  to  seek  help  and  more  likely  to  engage  in  secretive  behaviors. 
Such  research  suggests  that  people  who  score  low  on  this  one  item  will  similarly  be  less  likely  to 
openly  discuss  mental  health  issues  with  someone  they  see  who  may  be  struggling.  This  theory 
regarding  our  proxy  measure  is  confirmed  by  the  item  having  significant  correlations  with 
confidence  (r  =  .23,  p  =  .003),  knowledge  (r  =  .28,  p  =  .000),  and  practice  behaviors  scales  (r  = 

.28,  p  =  .000). 


Quick  Poll.  The  quick  poll  anonymous  survey  for  KSARNG  service  members  included  15  Likert 
scaled  items  on  a  scale  from  strongly  disagree  to  strongly  agree.  These  15-items  were  factor 
analyzed  following  the  same  procedures  as  the  CFR  survey  to  identify  the  best  functioning 
items  and  group  the  items  into  psychometrically  sound  scales  for  subsequent  analysis.  Based 
on  this  analysis  four  scales  were  identified — stress,  coping  resources,  knowledge  of  resources, 
and  opinions  towards  help  seeking — as  well  as  the  single  item  proxy  for  stigma.  Additional 
preliminary  evidence  of  construct  validity  was  confirmed  by  moderately  strong  and  significant 
correlations  between  all  scales  and  the  stigma  item  (r-values  between  .2  and  .5). 

Stress.  Stress  was  assessed  with  two  items:  (1)  military  and/or  job  related  stress;  and  (2)  family 
stress.  Reliability  is  marginally  acceptable  for  evaluation  purposes  (a  =  .66),  though  the 
relatively  low  alpha  is  likely  a  factor  of  the  scale  only  employing  two  items. 

Coping.  Coping  resources  assessed  a  person’s  perception  of  the  availability  of  resources  to 
help  them  cope  with  mental  or  emotional  challenges  and  their  perception  of  the  availability  of 
helping  support.  The  four  items  included:  (1)  having  positive  coping  strategies  to  deal  with 
stress;  (2)  peers  in  the  KSARNG  having  positive  coping  strategies  available;  (3)  having  a 
support  network  to  help  solve  problems;  and  (4)  peers  in  the  KSARNG  having  a  support 
network  to  help  solve  problems.  Reliability  for  this  scale  was  sufficient  (a  =  .79). 

Knowledge.  Knowledge  of  resources  for  mental  health  treatment  was  assessed  with  five  items, 
including  knowledge  of  (1 )  where  to  refer  someone  if  they  ask  for  help;  (2)  mental  health 
resources  within  the  National  Guard;  (3)  mental  health  resources  in  the  community  outside  of 
the  National  Guard;  (4)  crisis  resources  in  the  community;  and  (5)  at  least  two  trusted  resources 
to  refer  a  friend  or  family  member  to.  Reliability  for  this  scale  was  strong  (a  =  .86). 

Opinions.  Opinions  towards  help  seeking  was  assessed  with  three  items:  (1)  it’s  good  for 
someone  experiencing  stress  or  mental  health  issues  to  seek  help;  (2)  believing  counseling  is 
helpful;  (3)  it’s  helpful  for  people  experiencing  stress  or  mental  health  issues  to  use  the  available 
resources.  Reliability  for  this  scale  was  strong  (a  =  .82). 

Stigma.  Stigma  was  again  assessed  using  a  one-item  proxy  for  devaluation:  “People  are 
generally  supportive  when  a  person  is  experiencing  stress  and  related  mental  health  issues.” 
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Urban-Rural  Group  Equivalence.  Table  1  displays  the  means  and  standard  deviations  for  the 
urban  and  rural  groups  on  stigma,  attitudes,  coping  resources,  and  knowledge  at  pre-test,  and 
table  2  displays  the  t-test  results.  No  significant  differences  were  evident  between  the  urban  and 
rural  armories  prior  to  intervention  on  stigma,  f(268)  =  .68,  p  =  .50;  attitudes  towards  help 
seeking,  f(269)  =  -.83,  p  =  .40;  coping  resources,  f(269)  =  1.14,  p  =.26;  and  knowledge  of 
resources  f(269)  =  -1.75,  p  =.08).  These  results  indicate  that  the  Urban  and  Rural  armories  do 
not  differ  statistically  on  key  variables  which  could  distort  the  results  if  not  accounted  for.  Given 
the  lack  of  statistical  difference,  we  can  confidently  proceed  with  analyzing  the  data  set  without 
needing  to  account  for  urban  and  rural  differences. 


Table  1.  Pre-Intervention  Quick  Poll  means  and  standard  deviations  for  stigma,  attitudes, 
coping,  and  knowledge. _ 


Urban  /  Rural 

N 

Mean 

Std.  Deviation 

Stigma 

Urban 

108 

3.58 

1.02 

Rural 

162 

3.50 

.97 

Urban 

108 

3.95 

.82 

Attitudes  Toward  Help  Seeking 

Rural 

163 

4.03 

.67 

Urban 

108 

3.71 

.66 

Coping  and  Support  Resources 

Rural 

163 

3.61 

.74 

Urban 

108 

3.63 

.73 

Knowledge  of  Resources 

Rural 

163 

3.79 

.75 

Table  2.  f-test  results  for  pre-intervention  Quick  Poll  stigma,  attitudes,  coping,  and  knowledge 


t 

df 

Sig.  (2-tailed) 

Stigma 

.68 

268 

.50 

Attitudes  Toward  Help  Seeking 

-.83 

269 

.40 

Coping  and  Support  Resources 

1.14 

269 

.26 

Knowledge  of  Resources 

-1.75 

269 

.08 

Fidelity  Checks.  Table  3  displays  the  means  and  standard  deviations  for  the  behaviors, 
confidence,  knowledge,  attitudes,  and  stigma  at  pre-  and  post-training,  and  table  4  displays  the 
t-test  results  and  effect  sizes.  Tests  of  program  manipulation  and  fidelity  confirmed  that  the 
military  MHFA  training  resulted  in  significant  positive  gains  in  CFRs  likelihood  of  using  MHFA 
practices,  f(61)  =  -8.43,  p  <  .00;  confidence  or  self-perceived  ability  to  help  people  in  need, 
f(61)  =  -10.74,  p  <  .00;  knowledge  of  mental  health  resources,  f(61)  =  -9.87,  p  <  .00;  and 
attitudes  towards  help  seeking,  f(61)  =  -5.29,  p  <  .00.  Additionally,  CFRs  exhibited  significant 
decreases  in  stigma,  f(61)  =  -2.51,  p  =  .02.  The  effect  sizes  for  the  trainings  impact  on  practice 
behaviors  ( d  =  -2.16),  confidence  ( d  =  -2.75),  knowledge  ( d  =  -2.53),  and  attitudes  ( d  =  -1.35) 
are  all  very  large  based  on  Cohen’s  convention  of  effect  sizes  larger  than  .80  being  considered 
large.  The  effect  size  for  stigma  ( d  =  -.64)  was  also  meaningful  based  on  the  convention  of 
effect  sizes  between  .5  and  .8  being  considered  moderate. 

These  results  confirm  prior  research  indicating  the  MHFA  is  an  effective  method  for  increasing 
trainees  knowledge  of  mental  health,  positive  attitudes  towards  help  seeking,  and  helping 
behaviors,  while  also  reducing  stigma  towards  mental  illness  1’4.  These  results,  therefore, 
suggest  that: 
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(1)  the  adaptation  of  traditional  MHFA  for  military  populations  was  successful  in  replicating 
the  foundational  impacts  of  MHFA; 

(2)  the  trainings  were  implemented  with  sufficient  fidelity  to  replicate  expect  results;  and 

(3)  that  observed  positive  differences  between  intervention  and  control  groups  can  be 
attributed  to  the  manipulation. 

Table  3.  Descriptive  statistics  for  CFRs  pre-  and  post-training  on  practice  behaviors,  confidence, 


knowledge,  and  stigma. 


N 

Mean 

Std.  Deviation 

Behaviorl 

62 

3.98 

.58 

Behavior2 

4.60 

.46 

Confidencel 

62 

3.52 

.60 

Confidence2 

4.32 

.42 

Knowledgel 

62 

3.61 

.74 

Knowledge2 

4.49 

.47 

Attitudel 

62 

3.93 

.63 

Attitude2 

4.22 

.60 

Stigmal 

62 

3.05 

.90 

Stigma2 

3.42 

1.00 

Table  4.  f-test  results  and  effect  sizes  for  CFRs  pre  and  post-training 

Mean 

Difference 

t 

df 

Sig.  (2- 
tailed) 

Effect  Size 

Behaviorl  -  Behavior2 

-.62 

-8.43 

61 

.000 

-2.16 

Confidencel  -  Confidence2 

-.81 

-10.74 

61 

.000 

-2.75 

Knowledgel  -  Knowledge2 

-.88 

-9.87 

61 

.000 

-2.53 

Attitudel  -Attitude2 

-.30 

-5.29 

61 

.000 

-1.35 

Stigmal  -  Stigma2 

-.37 

-2.51 

61 

.015 

-.64 

Final  data  analyses 

Final  data  analyses  were  conducted  to  develop  preliminary  evidence  of  the  efficacy  of  the 
program  at  both  individual  and  community  levels.  We  conducted  a  series  mixed  between-within 
subjects  ANOVAs  comparing  the  intervention  and  control  group  on  CFR  results  longitudinally  at 
three  time  points:  pre-test,  at  4-months,  and  at  8-months.  Mixed  ANOVAs  and  examination  of 
descriptive  statistics  provide  simple  analytic  solutions  for  identifying  trends  and  preliminary 
evidence  of  efficacy,  as  well  as  expected  effect  sizes,  in  preparation  for  a  more  robust  study.  A 
mixed  ANOVA  as  implemented  here  tests  the  null  hypothesis  that  there  is  no  significant 
interaction  between  time  (three  repeated  measures)  and  intervention  grouping  (intervention  vs. 
control)  on  the  identified  dependent  variables.  In  addition  to  the  mixed  ANOVAs,  we  also 
conducted  repeated  measures  ANOVAs  on  data  from  CFRs  who  received  the  training  to 
confirm  whether  the  intervention  resulted  in  significant  changes  within  the  intervention  group. 
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The  analysis  of  CFR  results  was  conducted  prior  to  an  analysis  of  quick  poll  data  because  the 
CFR  results  are  the  most  proximate  to  the  intervention.  We  conducted  a  2  x  3  factorial  ANOVA 
with  the  quick  poll  data  to  test  for  community-level  outcomes  as  an  interaction  between  time  (3- 
levels:  pre-test,  4-months,  8-months)  and  armory  group  (2  levels:  intervention  (intervention  and 
control  armories).  Any  results  observed  in  the  quick  poll  data  must  be  interpreted  within  the 
context  of  CFR  results  as  these  outcomes  are  distal,  community-level  impacts  and  depend  on 
successful  results  at  the  individual  CFR  level  to  be  theoretically  linked  to  the  intervention. 

Community  First  Responder  Intervention  Effects.  Table  5  displays  the  results  from  the  mixed 
between-within  subjects  ANOVA  comparing  intervention  and  control  groups  on  behavior, 
confidence,  knowledge,  attitudes,  and  stigma  across  three  time  points  (pre-intervention,  4- 
months  post  intervention,  and  8-months  post-intervention).  Main  effects  for  time  (repeated 
measures)  for  confidence,  knowledge,  attitudes,  behaviors,  and  stigma  are  reported  in  Table  5, 
but  are  not  of  interest  for  the  study  as  they  test  for  change  over  time  across  the  entire  sample 
without  distinguishing  between  intervention  and  control  groups.  The  interaction  effects  between 
time  and  intervention  group  on  knowledge,  attitudes,  and  stigma  were  all  non-significant.  For 
knowledge,  attitudes  and  stigma  the  observed  effect  size  was  small  but  meaningful  ( q2=  .03  for 
knowledge;  rj2=  .02  for  attitudes;  q2=  .02  for  stigma).  There  was  a  non-significant  interaction 
effect  between  time  and  intervention  group  for  confidence  that  despite  non-significance  was 
trending  towards  significance  and  displayed  a  meaningful  and  moderate  effect  size,  F  (2)  = 
2.98,  p  =  .06,  rj2=  .06.  Finally,  there  was  a  significant  interaction  effect  between  time  and 
intervention  group  for  practice  behaviors,  F  (1 .75)  =  3.94,  p  =  .03,  q2  =  .07,  indicating  that  the 
intervention  had  an  observable  and  meaningful  impact  on  the  likely  hood  of  using  MHFA 
practices. 


Table  5.  Main  effects  of  time  and  interaction  effects  of  time  by  intervention  group  for  mixed 
between-within  subjects  ANOVA  of  CFR  results. _ 


Measure 

F 

df 

Sig. 

Partial 

Eta 

Squared 

Observed 

Power3 

Time 

Confidence 

14.36 

2 

.00 

.22 

.99 

Knowledge 

10.40 

2 

.00 

.17 

.99 

Attitudes 

1.93 

2 

.15 

.04 

.39 

Behaviors'3 

1.84 

1.75 

.17 

.04 

.34 

Stigma 

3.38 

2 

.04 

.06 

.63 

Time  *  IntGroup 

Confidence 

2.98 

2 

.06 

.06 

.57 

Knowledge 

1.61 

2 

.21 

.03 

.33 

Attitudes 

.83 

2 

.44 

.02 

.20 

Behaviors'3 

3.94 

1.75 

.03 

.07 

.64 

Stigma 

1.23 

2 

.30 

.02 

.26 

a.  Computed  using  alpha  =  .05 
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b.  Results  for  Behavior  apply  the  Greenhouse-Geisser  correction  for  violations  of  sphericitiy. 
Other  variables  did  not  violate  sphericity  and  the  results  reported  do  not  employ  the 
correction. 

The  small  but  meaningful  effect  size  for  knowledge,  attitudes,  and  stigma  indicates  that 
improved  measurement  precision  and  a  greater  sample  size  could  lead  to  identification  of 
significant  intervention  effects  over  time.  Likewise  for  confidence  the  effect  size  is  moderate 
indicating  that  an  adequately  powered  randomized  controlled  trial  would  likely  demonstrate  a 
significant  positive  effect  of  the  intervention  over  time  compared  to  the  control  group. 

Table  6  (placed  in  the  Appendix  due  to  the  length  of  the  table)  displays  the  descriptive  statistics 
and  Figure  1  displays  the  estimated  marginal  means  across  each  time  point  comparing 
intervention  and  control  groups  for  each  variable.  It  is  critical  to  turn  to  the  means  to  determine 
the  direction  of  change  In  order  to  interpret  the  results.  In  the  case  of  practice  behaviors,  the 
means  for  the  intervention  group  rises  dramatically  from  pre-intervention  to  4-months  post  and 
then  remains  stable  through  8-months  post  intervention.  In  contrast,  the  control  group  actually 
begins  with  higher  mean  practice  behaviors  than  the  intervention  group,  but  the  mean  scores 
are  fairly  stable  with  a  slight  decline  over  time.  At  8  months  post  intervention  the  intervention 
groups  mean  practice  behaviors  are  greater  than  the  control  group.  The  visual  representation  of 
these  differences  provided  in  the  chart  of  practice  behaviors  in  Figure  1  shows  just  how 
dramatically  different  a  pattern  of  change  there  is  between  the  intervention  and  control  group 
over  time.  These  results  with  the  statistically  significant  ANOVA  indicate  that  the  intervention 
positively  impacted  CFRs  MHFA  practice  behaviors  relative  to  a  control  group. 

For  confidence,  the  trend  is  similar  for  the  intervention  group  with  means  rising  from  time  1  to 
time  2,  then  remaining  relatively  stable  to  time  3.  For  the  control  group,  confidence  rises  from 
time  1  to  time  2,  but  then  falls  from  time  2  to  time  3.  As  with  practice  behaviors,  these  results 
point  towards  a  positive  intervention  impact  on  confidence  when  compared  to  a  control  group. 

For  resource  knowledge,  the  means  for  both  the  intervention  and  control  group  rise  from  time  1 
to  time  2,  but  then  are  relatively  stable.  For  attitudes,  both  groups  start  with  similar  scores  and 
conclude  with  similar  scores,  with  the  intervention  group’s  attitudes  improving  more  immediately 
than  the  control  group.  These  results,  coupled  with  the  non-significant  ANOVAs,  indicate  that 
there  may  not  be  an  intervention  effect  for  knowledge  of  resources  and  attitudes  towards  help 
seeking  when  compared  to  a  control  group.  For  knowledge  and  attitudes,  the  control  group  is 
showing  similar  change  over  time,  suggesting  that  there  may  be  some  other  confounding 
variable  in  the  KSARNG  that  is  driving  change  in  these  variables.  For  example,  if  there  was  an 
effort  external  to  this  study  to  increase  awareness  of  mental  health  resources,  then  both 
knowledge  and  attitudes  may  be  impacted  in  all  communities  irrespective  of  this  intervention.  In 
these  cases,  we  cannot  definitively  conclude  what  the  cause  of  the  observed  changes  is. 

With  stigma,  the  intervention  group’s  mean  score  increases  steadily  over  time  indicating  a 
reduction  in  stigma,  but  the  intervention  group  starts  off  with  greater  stigma  towards  mental 
illness  than  the  control  group.  In  contrast,  the  control  group’s  mean  score  decreases  a  small 
amount  at  each  time,  suggesting  a  possible  trend  towards  increasing  stigma  for  CFRs  who  did 
not  receive  the  intervention.  Despite  the  non-significant  ANOVA,  the  trend  displayed  in  the 
growth  chart  for  stigma  suggests  that  the  intervention  is  having  a  positive  impact  on  reducing 
stigma  despite  a  trend  in  the  control  group  of  increased  stigma  over  time.  In  other  words,  the 
intervention  may  be  turning  the  curve  completely  on  stigma — whereas  the  trend  in  the  KSARNG 
without  the  intervention  is  increasing  stigma,  this  intervention  may  actually  switch  the  direction 
of  change  towards  reduced  stigma. 
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Estimated  Marginal  Means  Estimated  Marginal  Means  Estimated  Marginal  Means 


Visual  inspection  of  the  growth  charts  for  each  variable  confirms  that  there  are  differences  in 
how  the  intervention  group  changed  on  each  variable  compared  to  the  control  group. 


Figure  1 .  Growth  charts  for  each  variable  of  change  overtime  comparing  intervention  and  control 
groups. 

Estimated  Marginal  Means  of  Confidence  Estimated  Marginal  Means  of  Knowledge 


Estimated  Marginal  Means  of  Attitudes  Estimated  Marginal  Means  of  Behaviors 


Estimated  Marginal  Means  of  Stigma 


Although  the  mixed  ANOVAs  did  not  provide  conclusive  evidence  of  the  intervention’s  impact, 
the  effect  sizes  and  descriptive  statistics  and  charts  indicate  a  trend  towards  efficacy.  To  further 
test  whether  the  positive  trends  of  CFRs  improving  on  each  variable  over  time  represent 
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meaningful  and  significant  differences,  we  conducted  repeated  measures  ANOVAs  on  each 
variable  over  time  for  only  the  intervention  group.  As  the  response  rate  for  CFRs  in  the 
intervention  condition  was  strong,  the  resultant  power  of  these  analyses  is  likewise  strong. 

Table  7  displays  the  results  of  the  one-way  repeated  measures  ANOVAs  for  CFR  trainees  on 
confidence,  knowledge,  attitudes,  behaviors,  and  stigma.  The  repeated  measure  ANOVA  tests 
the  null  hypothesis  that  there  is  no  difference  across  the  repeated  measures.  For  all  variables 
except  attitudes,  there  were  statistically  significant  positive  effects  over  time.  Post  hoc  tests 
(Table  8)  comparing  each  individual  time  point  to  the  other  two  (i.e.,  t1-t2,  t2-t3,  and  t1-t3), 
indicate  that  there  are  statistically  significant  gains  in  confidence,  knowledge,  behaviors,  and 
stigma  from  pre-intervention  to  4  months  post  intervention  and  from  pre-intervention  to  8  months 
post  intervention,  with  no  statistically  significant  difference  between  4  months  and  8  months 
post.  These  results  indicate  that  the  CFRs  showed  improvements  in  their  confidence, 
knowledge,  and  behaviors  and  decreases  in  stigma  from  pre-intervention  to  post-intervention, 
with  those  changes  being  sustained  and  stable  through  to  8-months  post-intervention. 

For  attitudes,  on  the  other  hand,  there  were  no  significant  effects  detected.  Although  difficult  to 
know  exactly  why  this  is  the  case,  we  believe  that  there  may  be  a  bit  of  a  ceiling  effect.  This  is  to 
say,  that  attitudes  towards  mental  health  treatments  were  positive  to  begin  with,  and  did  not  shift 
dramatically  over  the  course  of  the  study.  It  may  be  that  people  generally  believe  that  mental 
health  treatment  works,  and  that  receiving  the  m-MHFA  intervention  does  not  change  this 
already  positive  attitude. 

In  assessing  pilot  study  data  it  is  further  paramount  to  assess  the  effect  sizes  in  order  to 
properly  design  a  fully  powered  study.  For  confidence  (q2  =  .49),  knowledge  {q2  =  .39), 
behaviors  (q2  =  .27),  and  stigma  (q2  =  .16)  the  effect  sizes  are  large,  with  conventions  indicating 
that  /72  >  .  1 3  is  a  large  and  meaningful  effect.  Although  there  was  no  significant  effect  detected 
on  attitudes,  the  effect  size  is  non-the-less  moderate  with  a  low  observed  power,  suggesting 
that  a  better  powered  study  and/or  greater  measurement  precession  may  lead  to  a  detectable 
effect  on  attitudes  towards  help  seeking  as  well. 


Table  7.  One-way  repeated  measures  ANOVAs  for  CFR  trainees  at  pre-intervention,  4  months 
post,  and  8  months  post _ 


Measure 

F 

df 

Sig. 

Partial  Eta 
Squared 

Observed 

Power3 

Confidence 

36.95 

2 

.00 

.49 

1.00 

Knowledge 

24.26 

2 

.00 

.39 

1.00 

Attitudes 

2.32 

2 

.11 

.06 

.46 

Behaviors 

13.95 

2 

.00 

.27 

.99 

Stigma 

7.46 

2 

.00 

.16 

.93 

a.  Computed  using  alpha  =  .05 
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Table  8.  Post-hoc  comparisons  with  Bonferroni  correction 


Mean 

Measure 

(1)  Time 

(J)  Time 

Differenc 

Std.  Error 

Sig.b 

e  (l-J) 

2 

-.71* 

.10 

.00 

Confidence 

1 

3 

-.69* 

.10 

.00 

1 

.71 

.10 

.00 

2 

3 

.02 

.08 

1.00 

2 

-.63* 

.11 

.00 

1 

Knowledge 

3 

co 

CD 

.12 

.00 

2 

1 

.63* 

.11 

.00 

3 

.00 

.08 

1.00 

2 

-.16 

.08 

.14 

Attitudes 

1 

3 

-.15 

.09 

.32 

1 

.16 

.08 

.14 

2 

3 

.02 

.08 

1.00 

2 

-.42* 

.09 

.00 

1 

* 

Behaviors 

3 

-.37 

.10 

.00 

. 

1 

.42 

.09 

.00 

2 

3 

.06 

.07 

1.00 

2 

-.41* 

.15 

.03 

1 

Stigma 

3 

-.62* 

.15 

.00 

2 

1 

.41* 

.15 

.03 

3 

-.21 

.18 

.82 

Based  on  estimated  marginal  means 
*  The  mean  difference  is  significant  at  the  .05  level, 
b.  Adjustment  for  multiple  comparisons:  Bonferroni. 


Quick  Poll  Results.  To  test  for  potential  community-level  impacts,  we  conducted  2x3  factorial 
ANOVA  s  of  stress,  coping  resources,  knowledge,  attitudes,  and  stigma.  Data  for  the  quick  polls 
was  taken  from  random  samples  of  Guard  members  present  at  drills  during  data  collection — as 
such,  the  quick  polls  from  the  intervention  armory  do  not  provide  data  on  people  who  received 
the  training,  but  function  as  a  measurement  of  general  community  attitudes.  Time  (three 
measurements  at  pre-test,  4-months,  and  8-months)  and  Intervention  Group  (intervention  or 
control  armory)  were  the  fixed  factors,  and  the  interaction  of  time  and  intervention  group  tests 
the  null  hypothesis  that  there  is  no  significant  difference  on  the  dependent  variables  over  time 
and  across  intervention  groups.  The  test  asks  whether  the  training  of  CFRs  impacted  general 
community  attitudes  and  experiences. 

Table  9  displays  the  results  from  the  2  x  3  factorial  ANOVAs.  Significant  interaction  effects  were 
identified  for  coping,  F  (2)  =  3.02,  p  =  .05,  q2  =  .01 ,  knowledge,  F  (2)  =  7.93,  p  <  .00,  q2  =  .02, 
and  attitudes,  F  (2)  =  3.81,  p  =  .02,  q2  =  .01.  However,  inspection  of  the  interaction  charts 
comparing  change  over  time  between  the  intervention  and  control  groups  shows  that  the 
significant  effects  are  not  detecting  a  more  positive  intervention  effect  compared  to  the  control 
group.  In  complete  contrast  to  the  hypotheses,  the  interaction  charts  displayed  in  Figure  2  show 
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that  Guard  members  from  control  armories  appear  to  have  changed  more  for  the  positive  than 
those  from  intervention  armories.  For  this  reason,  we  can  reject  the  null  hypothesis  for  three  of 
the  variables,  but  have  not  been  able  to  identify  any  community-level  intervention  effects.  The 
changes  over  time  may  well  indicate  a  general  improvement  in  community  climate  towards 
mental  illness  in  the  KSARNG  over  the  course  of  the  study. 

Table  9.  Interaction  effects  for  Intervention  Group  by  Time  for  Quick  Poll  2x3  factorial  ANOVA 

Partial 

df  F  Sig.  Eta  Observed  Power3 

Squared 


Stress 

2 

.06 

.94 

.00 

.06 

Coping  and  Support 
Resources 

2 

3.02 

.05 

.01 

.58 

Knowledge  of  Resources 

2 

7.93 

.00 

.02 

.95 

Attitudes  towards  help 
seeking 

2 

3.81 

.02 

.01 

.69 

Stigma 

2 

.67 

.51 

.00 

.16 

a.  Computed  using  alpha  =  .  05 


Figure  2.  Interaction  charts  for  Intervention  Group  by  Time  for  Quick  Poll  2x3  factorial  ANOVA 

Estimated  Marginal  Means  of  Stress  Estimated  Marginal  Means  of  Coping  and  Support  Resources 
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Estimated  Marginal  Means  of  Knowledge  of  Resources  Estimated  Marginal  Means  of  Attitude  Toward  Help  Seeking 


Estimated  Marginal  Means  of  Stigma 


Present  data  to  Expert  Panel  and  Obtain  Feedback 

A  draft  of  the  research  report  was  sent  to  the  Expert  Panel  on  10/17/2014.  Comments  were 
received  back  from  the  Expert  Panel  on  10/23/2014,  and  feedback  was  incorporated  into  the 
final  report. 

Plan  larger  military  clinical  trial 

The  results  of  this  study  must  be  taken  as  preliminary  tests  of  both  efficacy  and  research 
feasibility.  In  particular,  the  study  is  underpowered  fortesting  for  interaction  effects  on  time  and 
intervention  group  across  all  dependent  variables  among  the  CFRs.  In  particular,  there  were 
insufficient  numbers  of  CFRs  at  follow-up  data  collection  to  draw  conclusive  conclusions  as  to 
whether  the  intervention  had  positive  impacts  when  compared  to  people  in  a  control  group. 
However,  one  way  repeated  measures  ANOVA  with  the  intervention  group  only  did  demonstrate 
significant  results.  Taken  together,  the  trends  in  the  mixed  ANOVAs  and  the  significant  results 
from  the  repeated  measures  ANOVAs,  the  results  show  promise  that  given  a  larger  study  we 
should  be  able  to  detect  significant  intervention  effects.  These  results  also  indicate  that  the 
study  design  is  also  a  feasible  method  for  detecting  intervention  results. 

Given  the  opportunity  for  a  larger  study,  we  would  recommend  a  greater  number  of  armories 
and  a  cluster  randomized  design.  With  20  or  more  sites  and  30-50  CFRS  trained  per  site  we 
would  have  sufficient  randomization  and  power  to  detect  intervention  effects.  Furthermore,  we 
would  recommend  modifying  the  survey  to  employ  more  standardized  measures,  in  particular 
for  assessing  stigma.  This  would  increase  the  measurement  precision  and  thereby  our  ability  to 
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detect  effects.  The  power  of  such  a  larger  study  would  also  permit  the  use  of  more  conclusive 
statistical  tests  to  compare  growth  in  variables  overtime  and  across  groups  using  hierarchical 
linear  modeling  techniques  controlling  for  the  cluster  randomized  design. 

Finally,  we  would  recommend  either  a  more  in-depth  participatory  research  design,  or  a  study 
with  regular  army  units.  In  order  to  test  this  intervention  for  Guard  Members  and/or  veterans,  we 
would  recommend  greater  community  engagement  with  non-Guard  community  members  joining 
the  advisory  committee  for  the  study.  Participatory  research  processes  would  enhance  our 
ability  to  engage  the  local  communities  surrounding  and  supporting  Guard  armories,  recruit 
more  natural  helpers  from  outside  of  the  Guard,  and  increase  community  penetration  of  the 
intervention.  Conversely,  a  study  with  regular  army  units  would  present  a  more  controlled 
environment  in  which  to  test  the  intervention.  Such  controlled  environments  would  allow  us 
great  ability  to  saturate  intervention  communities  and  track  potential  confounding  initiatives  in 
control  communities.  However,  there  are  significant  cultural  differences  and  barriers  potentially 
present  in  regular  units  with  respect  to  stigma  and  career  concern  that  may  operate  very 
differently  than  with  the  Guard.  Further,  the  original  goal  of  this  intervention  was  to  be  able  to 
leverage  the  wide-spread  MHFA  network  of  trainers  to  meet  the  specific  needs  of  Guard  and 
Reserve  members  and  veterans  who  are  living  in  nearly  every  community  in  the  country. 


KEY  RESEARCH  ACCOMPLISHMENTS 

•  Finalization  of  the  adapted  military  Mental  Health  First  Aid  curriculum. 

•  Roll  out  of  adapted  curriculum  to  the  nationwide  network  of  Mental  Health  First  Aid 
trainers. 

•  Demonstration  of  acceptability  of  the  intervention  among  National  Guard  members. 

•  Demonstration  of  feasibility  of  providing  trainings  to  National  Guard  members  and 
civilians. 

•  Demonstration  of  feasibility  of  research  methods  and  measures. 

•  Significant  improvement  that  was  sustained  to  8months  post  training  in  knowledge, 
attitudes,  behaviors,  confidence,  and  stigma  among  CFRs  trained  in  m-MHFA. 

•  Significant  intervention  effect  in  the  likelihood  of  using  MHFA  practice  behaviors  among 
CFRs  who  received  the  training  when  compared  to  the  control  group. 

•  Small  to  moderate  and  meaningful  effect  sizes  for  the  difference  between  CFRs  who 
received  the  training  and  the  control  group  indicating  meaningful  change  overtime  as  a 
result  of  the  intervention. 


CONCLUSION 

The  adaptation  process  for  the  Mental  Health  First  Aid  curriculum  implemented  during  the  first 
year  of  this  study  has  the  potential  for  significant  impact  on  the  military/veteran  population 
across  the  country.  The  National  Council  for  Behavioral  Health  manages  dissemination  of  the 
MHFA  curriculum  throughout  the  United  States.  There  are  now  more  than  30,000  instructors 
throughout  the  country.  Once  those  who  are  qualified  to  teach  the  adaptation  complete  the 
process  to  receive  this  designation,  there  is  the  potential  to  reach  a  significant  number  of 
service  members,  veterans,  and  their  families  nationwide.  The  military  Mental  Health  First  Aid 
curriculum  has  the  potential  to  complement  existing  military  programs  such  as  resilience 
training,  Combat  &  Operational  Stress  First  Aid,  and  suicide  prevention  gatekeeper  training  by 
providing  a  mental  health  literacy  component  that  is  currently  not  addressed. 

Despite  the  promise  of  m-MHFA  to  have  substantial  and  meaningful  impacts,  the  results  of  this 
study  remain  preliminary.  The  study  was  underpowered  to  detect  true  intervention  effects  due  to 
small  sample  sizes,  especially  within  the  control  group  at  follow-up  measurement  times.  A  larger 
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number  of  participants  would  provide  not  only  greater  power,  but  would  allow  for  more 
sophisticated  longitudinal  analyses  that  would  be  better  able  to  detect  group  differences  in 
change  over  time.  Furthermore,  the  study  design  was  not  fully  randomized — with  only  four 
armories  there  was  not  a  large  enough  number  of  clusters  for  the  cluster  randomization 
approach  to  be  meaningful.  From  a  statistical  standpoint  hierarchical  models  accounting  for  the 
clusters  demand  far  more  clusters  for  adequate  power.  Therefore,  we  caution  that  these  results 
be  taken  only  as  preliminary  evidence. 

We  were  unable  to  meaningfully  measure  community-level  impacts.  This  may  be  due  to 
inadequate  measurement  strategies  or  due  to  the  distal  nature  of  community-level  impacts. 
Given  the  training  of  a  relatively  small  number  of  people,  it  is  unlikely  to  see  changes  in  the 
larger  community’s  attitudes  and  resources.  The  observed  effect  sizes  for  the  trainees  relative 
to  the  control  group  were  all  small  to  moderate,  hence  it  is  logical  to  assume  that  such  small 
effects  among  25  people  per  community  would  result  in  even  smaller  community-levels  effects. 
Detecting  such  effects  would  take  greater  saturation  of  the  community  with  trained  CFRs  (that 
is,  more  trainings),  very  precise  and  reliable  measurement  techniques,  large  sample  sizes,  and 
observation  over  a  longer  period  of  time. 

One  troubling  aspect,  however,  of  the  community-level  outcomes  was  a  trend  showing  that  the 
control  group  had  equal  or  greater  gains  on  all  variables  of  interest.  This  suggests  that  other 
awareness  and  education  efforts  and  natural  changes  within  the  National  Guard  may  be  fully 
accounting  for  observed  growth  in  community  attitudes.  In  future  studies  it  will  be  critical  to  track 
and  measure  what  other  mental  health  interventions  are  going  on  over  the  course  of  the  study 
to  better  account  for  these  potential  confounds.  The  current  results,  though,  suggest  that  the  m- 
MHFA  intervention  may  be  no  more  effective  at  enhancing  community-level  outcomes  than 
other  ongoing  efforts. 

With  respect  to  the  more  proximate  outcomes  comparing  CFRs  who  received  the  training  to 
those  who  did  not,  we  have  more  promising  results  to  report.  Across  all  variables  we  measured 
from  pre-intervention  to  4-months  and  8-months  later,  the  intervention  group  displayed 
statistically  significant  improvements  and  moderate  to  large  effects.  When  compared  to  the 
intervention  group  though,  most  of  these  effects  were  no  longer  significant.  Again,  the  control 
group  also  showed  improvement  over  time  on  most  variables,  though  often  to  a  lesser 
magnitude.  Further,  the  control  group  in  general  had  greater  knowledge,  attitudes,  behaviors, 
and  confidence  and  less  stigma  relative  to  the  intervention  group  at  the  beginning  of  the  study. 
This  indicates  that  the  control  group  may  very  well  have  been  biased  and  that  the  randomization 
strategy  may  have  been  inadequate. 

The  final  sample  size  and  bias  in  the  control  group  were  likely  results  of  a  number  of 
methodological  challenges  we  encountered  in  carrying  out  the  study.  First,  the  original  proposed 
study  included  more  significant  community  engagement  with  the  communities  surrounding  the 
armories  and  stronger  recruitment  of  non-Guard  CFRs.  Ultimately,  the  funding  was  insufficient 
for  a  greater  community  engagement  effort,  which  lead  to  lower-that-hoped  for  recruitment  of 
non-Guard  participants,  as  well  as  resulted  in  an  inability  to  retain  control  group  participants  as 
we  were  unable  to  offer  any  incentive  for  them  to  stay  engaged  over  the  course  of  the  study. 
Second,  we  had  also  intended  to  collect  more  data  employing  well-researched  measurement 
scales,  as  well  as  administrative  data  on  mental  health  referrals  and  access  to  services. 
However,  through  negotiations  with  the  Guard  we  had  to  change  our  measurement  strategy  and 
lost  the  measurement  precision  of  using  a  more  in-depth  data  collection  survey.  We  also  ran 
into  numerous  delays  that  made  it  impossible  for  us  to  have  the  time  necessary  to  access 
administrative  data.  Finally,  the  Guard  also  disallowed  our  ability  to  offer  Guard  participants  a 
small  stipend  for  completing  surveys,  which  may  have  also  contributed  to  the  low  response 
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rates  at  4-  and  8-month  follow  up.  Future  research  will  need  to  carefully  take  into  account  the 
true  time  it  takes  to  work  with  military  structures  in  carrying  out  research,  as  well  as  budget  for 
stronger  community  engagement  and  research  participant  retention  strategies. 

Despite  the  challenges  due  to  small  samples,  new  measures  and  lack  of  true  randomization,  the 
study  proved  adequate  for  providing  preliminary  evidence  that  m-MHFA  is  an  impactful 
intervention.  First,  and  most  critically,  there  was  a  significant  intervention  effect  detected  for  the 
likelihood  that  a  CFR  would  use  appropriate  engagement,  support,  and  referral  practices  when 
identifying  someone  in  need  of  mental  health  support.  Second,  there  was  a  clear  trend  showing 
a  similar  effect  for  confidence  in  helping  someone  in  need.  And  third,  for  all  the  other  three 
measured  variables — knowledge  of  mental  health  resources,  attitudes  towards  help  seeking, 
and  stigma — there  were  positive  growth  trends  in  the  data  and  small  but  meaningful  effect  sizes 
that  clearly  indicate  that  with  more  power  (a  larger  number  of  participants  and  armories)  we 
would  expect  to  see  significant  intervention  effects.  These  results  can  be  interpreted  as  positive 
preliminary  evidence  of  program’s  effects. 

In  order  to  establish  program  efficacy  a  follow-up  cluster  randomized  study  with  a  larger  number 
of  clusters  (at  least  20  armories/communities)  and  a  larger  number  of  participants  (25-50  per 
site)  is  warranted.  We  anticipate  that  such  a  study  would  clearly  establish  m-MHFA’s  efficacy  at 
improving  mental  health  literacy,  mental  health  first  aid  practice  behaviors,  and  attitudes 
towards  help  seeking  and  would  reduce  stigma.  In  a  follow-up  study  we  would  recommend 
slightly  modifying  the  measures  for  CFRs  to  improve  measurement  precision  and  include  other 
measures  of  mental  health  status,  as  well  as  developing  stronger  methods  for  assessing 
community  level  impacts.  Given  our  experience  on  this  study,  however,  we  would  still 
recommend  keeping  the  surveys  brief  so  as  to  be  feasible  and  acceptable  to  conducting 
research  with  military  populations.  To  assess  community  level  impacts,  it  will  be  critical  to 
access  administrative  records,  public  health  systems  data,  and  electronic  health  data  from 
major  providers  in  the  served  communities.  Polling  the  armories’  general  population  did  not 
prove  useful  in  this  study,  but  if  targeted  towards  brief  measures  of  attitudes  and  stigma  may 
still  be  a  useful  approach  if  we  can  track  individuals  over  time  and  have  greater  control  over  the 
study  conditions. 

Finally,  it  will  also  be  essential  to  employ  stronger  methods  for  retaining  research  participants  in 
any  future  study.  In  the  current  pilot  study,  attrition  among  the  control  group  had  detrimental 
effects  on  the  retention  of  participants  and  statistical  power,  as  well  as  being  a  likely  source  of 
bias.  In  this  study  we  were  prohibited  from  offering  even  small  honorariums  to  National  Guard 
participants  for  completing  the  surveys,  a  technique  that  is  standard  practice  in  similar  studies 
and  greatly  improves  return  rates  for  follow-up  surveys.  Further,  in  a  larger  study  we  would 
recommend  a  wait-list  control  design  so  that  every  CFR,  including  those  randomized  to  the 
control  group,  would  receive  the  training,  thereby  increasing  buy-in  and  commitment  from 
participants  in  the  control  group. 

In  conclusion,  we  feel  that  military  MHFA  is  a  promising  intervention  for  increasing  knowledge  of 
mental  health  and  related  resources,  increasing  positive  attitudes  towards  help  seeking, 
decreasing  stigma  towards  mental  illness,  and  increasing  the  likelihood  that  people  will  use  of 
effective  behaviors  when  supporting  a  person  in  need.  Our  evidence  casts  doubt  on  whether  it 
is  feasible  to  measure  more  distal  community-level  outcomes  and  suggests  that  natural 
changes  in  community  climate  towards  mental  illness  may  have  equal  or  more  profound  impacts 
on  community  attitudes  and  availability  of  resources  than  this  intervention.  However,  for  shifting 
people’s  attitudes  directly,  the  intervention  appears  to  be  an  impactful  training  with  a  substantial 
dissemination  system  in  place  making  it  a  useful,  ready-to-go  intervention. 
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PUBLICATIONS,  ABSTRACTS  AND  PRESENTATIONS 


•  Mental  Health  First  Aid  for  Veterans  Press  Conference  and  Stakeholder  Briefing  - 

National  Press  Club,  April  24,  2014 

•  Defense  Centers  of  Excellence,  Deployment  Health  Clinical  Center  (DHCC)  Mental 
Health  First  Aid  panel,  July  9,  2014 

INVENTIONS,  PATENTS  AND  LICENSES 

Nothing  to  report. 

REPORTABLE  OUTCOMES 

•  Demonstration  of  acceptability  of  the  intervention  among  National  Guard  members. 

•  Significant  improvement  that  was  sustained  to  8-months  post  training  in  knowledge, 
attitudes,  behaviors,  confidence,  and  stigma  among  CFRs  trained  in  mMHFA. 

•  Significant  intervention  effect  in  the  likelihood  of  using  MHFA  practice  behaviors  among 
CFRs  who  received  the  training  when  compared  to  the  control  group. 

•  Small  to  moderate  and  meaningful  effect  sizes  for  the  difference  between  CFRs  who 
received  the  training  and  the  control  group  indicating  meaningful  change  overtime  as  a 
result  of  the  intervention. 

•  No  significant  changes  in  general  National  Guard  armory  attitudes  and  resources. 


OTHER  ACHIEVEMENTS 

Nothing  to  report. 
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APPENDICES 


Community  First  Responder  survey 
Quick  poll  survey 

Descriptive  Statistics  From  CFR  Surveys 


Table  6.  Descriptive  statistics  of  CFR  Surveys 


IntGroup 

Mean 

Std.  Deviation 

N 

Confidence:  Pre¬ 

Intervention 

3.48 

.70 

39 

intervention 

Control 

3.67 

.33 

13 

Total 

3.52 

.63 

52 

Confidence:  4-Month 

Intervention 

4.19 

.47 

39 

Follow-up 

Control 

4.10 

.54 

13 

Total 

4.16 

.49 

52 

Confidence:  8-Month 

Intervention 

4.17 

.74 

39 

Follow-up 

Control 

3.83 

.93 

13 

Total 

4.08 

.79 

52 

Resource  Knowledge: 

Intervention 

3.55 

.79 

39 

Pre-intervention 

Control 

3.83 

.50 

13 

Total 

3.62 

.74 

52 

Resource  Knowledge: 

Intervention 

4.18 

.73 

39 

4-Month  Follow-up 

Control 

4.17 

.45 

13 

Total 

4.18 

.67 

52 

Resource  Knowledge: 

Intervention 

4.18 

.79 

39 

8-Month  Follow-up 

Control 

4.06 

1.06 

13 

Total 

4.15 

.86 

52 

Attitudes:  Pre¬ 

Intervention 

3.97 

.64 

39 

intervention 

Control 

3.92 

.63 

13 

Total 

3.96 

.63 

52 

Attitudes:  4-Month 

Intervention 

4.13 

.51 

39 

Follow-up 

Control 

3.92 

.68 

13 

Total 

4.08 

.56 

52 

Attitudes:  8-Month 

Intervention 

4.11 

.77 

39 

Follow-up 

Control 

4.13 

1.01 

13 

Total 

4.12 

.83 

52 

Practice  Behaviors: 

Intervention 

3.94 

.72 

39 

Pre-intervention 

Control 

4.45 

.50 

13 

Total 

4.06 

.70 

52 

Practice  Behaviors:  4- 

Intervention 

4.36 

.56 

39 
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IntGroup 

Mean 

Std.  Deviation 

N 

Confidence:  Pre¬ 

Intervention 

3.48 

.70 

39 

intervention 

Control 

3.67 

.33 

13 

Total 

3.52 

.63 

52 

Confidence:  4-Month 

Intervention 

4.19 

.47 

39 

Follow-up 

Control 

4.10 

.54 

13 

Total 

4.16 

.49 

52 

Confidence:  8-Month 

Intervention 

4.17 

.74 

39 

Follow-up 

Control 

3.83 

.93 

13 

Total 

4.08 

.79 

52 

Resource  Knowledge: 

Intervention 

3.55 

.79 

39 

Pre-intervention 

Control 

3.83 

.50 

13 

Total 

3.62 

.74 

52 

Resource  Knowledge: 

Intervention 

4.18 

.73 

39 

4-Month  Follow-up 

Control 

4.17 

.45 

13 

Total 

4.18 

.67 

52 

Resource  Knowledge: 

Intervention 

4.18 

.79 

39 

8-Month  Follow-up 

Control 

4.06 

1.06 

13 

Total 

4.15 

.86 

52 

Attitudes:  Pre¬ 

Intervention 

3.97 

.64 

39 

intervention 

Control 

3.92 

.63 

13 

Total 

3.96 

.63 

52 

Month  Follow-up 

Control 

4.44 

.48 

13 

Total 

4.38 

.54 

52 

Practice  Behaviors:  8- 

Intervention 

4.30 

.74 

39 

Month  Follow-up 

Control 

4.23 

1.07 

13 

Total 

4.29 

.82 

52 

Stigma:  Pre¬ 

Intervention 

2.90 

.94 

39 

intervention 

Control 

3.62 

.87 

13 

Total 

3.08 

.97 

52 

Stigma:  4-Month 

Intervention 

3.31 

1.00 

39 

Follow-up 

Control 

3.54 

1.05 

13 

Total 

3.37 

1.01 

52 

Stigma:  8-Month 

Intervention 

3.51 

.94 

39 

Follow-up 

Control 

3.85 

1.07 

13 

Total 

3.60 

.98 

52 

Military /Veteran  Mental  Health  First  Aid  PowerPoint  Slide  Deck 
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Military/Veteran  Mental  Health  First  Aid  Instructor  Notes 

SUPPORTING  DATA: 

Nothing  to  report. 
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Dennis  Mohatt,  MA 
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Kristin  Musch,  RN 
Jeremy  Vogt,  PhD 
Richard  Westphal,  PhD 
Nicola  Winkel,  MPA 
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